The decreasing cost of sensor hardware and the availability of vast communications networks have permitted monitoring the conditions of remotely located assets over a long time period, collecting enormous amounts of data. Examples of such monitoring systems abound, and include systems for monitoring the condition of transportation infrastructure including bridges, tunnels and highways, communications and power distribution infrastructure such as cables and switches, wind turbines, waterways, and equipment subsystems such as factory machines and agricultural equipment. In each case, general or special-purpose sensors produce measurement data that is collected by processors and analyzed using one or more data analysis algorithms.
A typical data monitoring system 100 for long-term monitoring is shown in FIG. 1. A data logger 110 is typically used to gather data from simple sensors 105, 106. As used herein, the term “sensor” means any device that performs a measurement of its environment and transmits a signal representing that measurement. Examples of sensors include distance sensors, temperature sensors, pressure sensors, strain gages, color sensors, liquid level sensors, cameras, microphones, chemical sensors, electrical sensors, speed, altitude or pitch sensors, GPS receivers, etc.
The data logger 110 may perform one or more functions on signals received from the sensors 105, 106. For example, the data logger may condition and compensate a signal for offset and gain, or may perform analog to digital conversion on a signal. The data logger may receive wireless signals from the sensors and convert them to terrestrial electrical or optical signals. The data logger may organize signals from multiple sensors and store data representing those signals in a temporary memory until the data is retrieved by a monitoring system. The data logger may associate sequential data from a sensor with timestamps to permit synchronization with data from other sources. The data logger may additionally calculate characteristics that are not directly measured by the sensors, but are instead determined by combining or compensating signals received from the sensors. The data logger typically includes a processor that may be queried by a monitoring system to retrieve stored data.
Data 115 from the data logger 110 are collected by the monitoring system 100 and stored into a measurement database 125. A system of measurement sensors such that described may generate a large volume of data; the measurement database must therefore be capable of efficiently storing and accessing such data. In practice, a column-oriented database management system architecture is frequently used.
The data may be transmitted and stored in real time as it is measured, or may be collected in small batches by the data logger and transmitted at regular time intervals or regular batch sizes to be stored by the monitoring system 100 in the database 125. The stored data may include data from multiple sensors, indexed according to a time line, or according to some other indexing variable.
The stored data 115 in the database 125 are constantly analyzed through one or more analysis methods performed by one or more data analysis software packages 130. For example, outlier detection may be performed on the data, or a state estimation model may be used to detect deviations from a normal state. Results of those analyses are then used to generate labels 140 that are indexed to the data 115. Each data analysis software package 130 or algorithm typically generates a particular set of labels 140 directed to a particular characteristic. The labels are stored in an event database 135 as events and may include indexing data such as time data or unique IDs for aligning the labels with the corresponding measurement data 115. The labels 140 annotate the data 115 by marking intervals or points in the data 115 where a certain error has been observed, a certain threshold has been exceeded, or some other event has occurred as determined by the data analysis software packages 130.
A data analysis software package 130 utilizes a particular set of data from the data logger 110. Often, that subset of data is only a small portion of the data generated by the logger over time. In a large system, the various algorithms used in the data analysis software packages 130 must parse through huge amounts of data generated by the data loggers to retrieve the data that is useful to a particular algorithm.
The monitoring system 100 is often deployed in a long-term setting IT framework that supports long-term monitoring. In those cases, it is desirable that the system be flexible enough to adapt over time to newly available sensing and logging technologies and new equipment models and vendors. The system should also adapt to new algorithmic developments. Currently available IT frameworks are vendor specific and not open for collecting data from sensing hardware of different providers. The frameworks are furthermore not sufficiently flexible to connect to sensing technologies not seen so far. In addition, existing frameworks only allow the creation of very basic algorithms such as rule-based or fuzzy logic-based methods and are not open to the integration of more sophisticated custom-made algorithms.
There is presently a need to overcome the above described limitations of existing IT solutions by facilitating the integration of a variety of different sensor types and data analysis methods. There is furthermore a need for the capability to extend the capabilities of a measurement system to function with new sensor inputs and new data analysis algorithms.